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In electronic holography, various methods have been considered for using multiple spatial light modulators 
(SLM) to increase the image size. In a previous work, we used a monochrome light source for a method that 
located an optical system containing lens arrays and other components in front of multiple SLMs. This paper 
proposes a colourization technique for that system based on time division multiplexing using laser light 
sources of three colours (red, green, and blue). The experimental device we constructed was able to perform 
video playback (20 fps) in colour of full parallax holographic three-dimensional (3D) images with an image 
size of 63 mm and a viewing-zone angle of 5.6 degrees without losing any part of the 3D image. 

Recently, expectations for ultra realistic video technologies have increased and a great deal of research is 
being performed to realize ultra high- definition or three-dimensional (3D) implementations. Another 
aspect of ultra realistic video research involves 3D video images that provide a sense of reality that makes 
it seem as if an object were actually present. There are numerous methods of displaying 3D video, which can be 
classified into two-view display 1 or multi-view display 2 ' 3 , volumetric display 4 , or spatial image reproduction 
display 5 ' 6 . Among these, the image reproduction 3D display method, in particular, can display easy-to-view 
3D images naturally in a similar method as when an object is being directly observed since it is a method that 
reconstructs the image spatially. Although image reproduction display can be achieved using integral pho- 
tographic and holographic methods, the holographic method is considered to be the ultimate 3D video display 
method since it can reproduce the wavefronts of the light reflected by an object and propagated through space. 

A wide range of research has been conducted concerning electronic holography-type 3D displays for repro- 
ducing 3D videos based on holographic principles. This includes research directed towards increasing the image 
size 7 and research directed towards real-time operation 8 . 

The important topics of investigation for this kind of 3D display are the image size S of the 3D image and the 
viewing-zone angle 0 indicating the range in which the 3D image can be seen. Naturally, it is desirable for both S 
and 0 to be large. If we denote the number of spatial light modulator (SLM) pixels here by N x in the horizontal 
direction and N y in the vertical direction, the pixel spacing byp x in the horizontal direction andp y in the vertical 
direction, and the wavelength of the reproduced light by 2, then the image size S x and viewing-zone angle 0 X in the 
horizontal direction are given as follows: 



--N x p x , 



0 X = 2 arcsin 



2px Px 



(1) 



(2) 



Therefore, it is apparent that the product S X 8 X is proportional to N x as shown in equation (3): 

s x 6 x =n x a, 



(3) 



equation (2) shows, viewing-zone angle 0 X and display bandwidth II p x of SLM are equivalent, so equation (3) also 
turn out to be space-bandwidth product. 
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Table 1 | System parameters 
Parameter 



Design value 



Wave length of lasers (^r, Xq, ^q) 

Number of pixels for each SLM (horizontal N x , vertical N y ) 
Pixel pitch for each SLM (horizontal p x/ vertical p y ) 
Image size for each SLM S' (horizontal S x , vertical S y ) 
Viewing-zone angle for each SLM (horizontal 6 Xf vertical 9 y ) 
Number of SLMs K (horizontal K x , vertical K y ) 
Gap between adjacent SLMs V (horizontal T x , vertical T y ) 

Parameters of lenses LjJ, lJ, and ±Lq (focal length fo, horizontal size £o, vertical size r]o) 
Parameters of lenses Lj, l}, and L 2 (focal length f], horizontal size £i, vertical size r]-\) 
Parameters of lens I2 (focal length $2, diameter §2) 
Parameters of lens L3 (focal length f^, diameter ^3) 
Hologram data update rate 



(633,532,473) [nm] 
(3840,2160) [pixels] 
(4.8, 4.8) [urn] 

Diagonal 2 1.1 (18.4, 10.4) [mm] 

(5.6, 2.8) [degrees] 

(3,3) 

(55.3,31.1) [mm] 
(1 10,50, 40) [mm] 

(440, 73.7,41.5) [mm] 

(800, 330) [mm] 
(200, 1 00) [mm] 
60 [fps] 



In other words, to simultaneously increase S x and 0 X , the number 
of pixels N x must increase. Note that we only provided an explana- 
tion for the horizontal direction here since the vertical direction is 
exactly the same. All subsequent explanations will also only be given 
for the horizontal direction. 

Although it is desirable to increase the number of pixels N x of an 
SLM, the maximum number is limited for a single SLM. Therefore, a 
space-division multiplexing method has been investigated for 
increasing the number of pixels using multiple SLMs and a time- 
division multiplexing method has been investigated for rapidly 
rewriting the hologram data that is displayed on an SLM to make 
it seem as if N x were increased. With the space-division multiplexing 
method, if we denote the number of pixels in the horizontal direction 
of each SLM by N x and denote the number of SLMs by K x > then N x = 
N X K X . Some time-division multiplexing methods that have been 
attempted use shutters 9,10 and others use galvano mirrors 11 . There 
is also a known technique that has devised a good way to use both 
space-division and time-division multiplexing together 12 . Since we 
want to discuss a method of using multiple SLMs here, we decided to 
deal with only a space-division multiplexing method. 

Various methods for increasing the viewing zone by using a space- 
division multiplexing method have been proposed 13 " 15 . These meth- 
ods are affected by the problem that part of the viewing-zone angle is 
missing because of the gaps between adjacent SLMs since it is phys- 
ically impossible to place the SLMs very closely together. Therefore, 
finding a means of eliminating this missing segment problem is a 
major issue. A method have been proposed which overcome this 
problem by using beam splitter and multiple SLMs 16 . However, view- 
ing-zone angle only scalable in horizontal direction, and each gaps 
between SLMs are necessary equal to width of SLM. 

Various methods of increasing the image size by using space- 
division multiplexing methods have also been proposed. Problems 
with these proposals are that they only provide parallax in the 
horizontal direction 1718 besides image size only scalable in hori- 
zontal direction 19 , or decrease display frame late which is inversely 




(a) 

Figure 1 | Experimental setup. An exterior view of the constructed system 



proportional to number of space-division multiplexing 20 . In addi- 
tion, finding a means of eliminating missing image segments is also 
a major issue for these methods of increasing the image size just like it 
is for the methods of increasing the viewing zone. 

In a previous work, we proposed a method of increasing the image 
size using space -division multiplexing by locating an optical system 
containing a lens array and other components in front of multiple 
SLMs 21 . This method not only can eliminate missing image segments, 
but can also control the balance between image size and viewing-zone 
angle by varying the focal length of lenses in the latter half within the 
optical system. In addition, it can exhibit parallax in both the hori- 
zontal and vertical directions unlike the methods mentioned earlier. 

However, the technique for colourizing the system which was 
adopted by this method was difficult because the optical system is 
too complex to use a combination of three different colour light 
sources 22 . In this paper, we propose a colourization technique for 
electronic holography that reconstructs 3D images at a video rate 
based on time division multiplexing 23 " 29 using three lasers (red, 
green, and blue) and a set of three revolving wheel shutters synchro- 
nized with 60 Hz driven SLMs. 

Results 

We constructed a device with the parameters shown in Table 1. 
This device, which uses an optical system and 9 SLMs, corre- 
sponds to a SLM with approximately 74,600,000 pixels. Figure 1 
and Video 1 show experimental setup. Figure 1(a) shows an exter- 
ior view of the constructed device, and Fig. 1(b) shows its light 
source parts. We performed the following experiments using holo- 
gram data created by computer generated hologram (CGH) tech- 
nology. Note that in this device, horizontal and vertical gap 
between adjacent SLMs T' x and T' y are three times the image size 
of each SLM S' x and S' y due to physical constraints of the SLMs. In 
other words, for a display area of one for the image part, the area 
of the missing segments is eight. Therefore, since the reproduced 




(b) 

(a) and its light source parts (b). H.S. took photographs. 
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Figure 2 | Experimental results of 3D objects located at different depth. Design values of the displayed objects (a); photographs that were taken 
focused on the "C" in the 30 mm front (b); and the "N", "I" and "T" in the 30 mm back from the hologram plane and that were taken from locations 2.8 
degrees to the right (c), directly in front (d), and 2.8 degrees to the left (e). H.S. took photographs. 



image cannot be directly observed straight on, an optical system 
like the one described here is required. 

We confirmed that objects with different depths could be recon- 
structed in colour by using time division multiplexing. Figure 2 
shows the experimental results. Figure 2(a) shows the design values 
of the displayed objects. The colours of the objects (blocks with a 
letter) are set as follows in the hologram data: "N" is green, "I" is 
yellow (red + green), "C" is cyan (green + blue), and "T" is magenta 
(red + blue). Figures 2(b) and 2(d) show photographs that were 
taken of the reconstructed objects focused on the "N", "I" and "T" 
located 30 mm behind the hologram plane and the "C" located 
30 mm in front of the plane, respectively. When the objects can be 
reconstructed at different depths, "C" should be blurry in Fig. 2(b) 
and "N", "I" and "T" should be blurry in Fig. 2(d). From the experi- 
mental results and the description above, it is apparent that objects 
with different depths can be reconstructed. Figures 2(c), 2(d), and 
2(e) show the views of the same displayed object from other angles. 
These figures show photographs that were taken of the reconstructed 
objects from locations 2.8 degrees to the right, directly in front, and 
2.8 degrees to the left, respectively. It is apparent that the recon- 
structed objects can be observed in the designed viewing zone and 
these missing segments that can be seen without the optical system 
are eliminated. We used method of computer generated hologram 



(CGH) to create hologram data for the earth and moon. Figure 3 
shows the results of a similar experiment using this data. 

We also confirmed that a video could be reproduced. Video 2 
shows the experimental results. The letter "N" located 30 mm 
behind the hologram plane gradually moves forward until it is 
30 mm in front of the plane and then gradually moves back until 
it is 30 mm behind the plane. The video sequence shows the 
letters "I", "C" and "T" also sequentially moving forward and 
back in a similar manner. This experiment confirmed that this 
video sequence can be converted to a frame rate (with one set 
of R, G, and B per frame) in time- division colour and played 
smoothly at a 20 frames per second (fps). From the experiments 
described above, it is apparent that a coloured moving image of 
3D objects having an image size 55.3 mm X 31.1 mm which is 
nine times that of a SLM, and a horizontal viewing- zone angle of 
5.6 degrees can be played with a frame rate of 20 fps using the 
proposed method. 

Figure 2, 3 and Video 2 show gaps between SLMs are eliminated. 
However boundaries of SLMs are observed. For example, observed 
boundaries are sparser in Fig. 3 (d) focused on earth, located far from 
the hologram plane, than boundaries in Fig. 3 (b), focused on moon, 
located near the hologram plane. And, difference of brightness 
between each small display area are also observed. 



Hologram plane 

Moorv-^ 




30 [mm] 



<y 80 [mm] 



(a) 



(b) 



(c) 




(e) 



Figure 3 | Experimental results of 3D objects located at different depth. Design values of the displayed objects (a); photographs that were taken 
focused on the moon in the 30 mm front (b); and the earth in the 1 10 mm front from the hologram plane and that were taken from locations 2.8 degrees 
to the right (c), directly in front (d), and 2.8 degrees to the left (e). H.S. took photographs. 
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We also performed a display experiment in which we used a 
method 30 to convert images captured by a 3D camera based on integ- 
ral photography (IP) to hologram data. IP is a method of capturing 
many angle of light ray information in space at once by using lens 
array 31 . The camera system consists of high-resolution (4K) camera 
and lens array (240 X 135 lenses, 0.8 mm pitch). Figure 4(a) shows 
the foregoing camera system and reconstructed 3D images which 
converted to hologram from captured IP image. We calculated wave- 
fronts propagation by IP image using computer to get hologram. We 
confirmed that the photographed images could be displayed by the 
foregoing system. Figures 4(b) and 4(c) show results of displaying 
hologram data which converted from images captured by the fore- 
going camera system. These pictures were taken focused on the 
panda, and angel in left which is positioned in front of the panda, 
respectively. From these results, we confirmed that depth and surface 
texture of 3D images are reproduced truly. 

Discussion 

A problem encountered when increasing the image size by using 
multiple SLMs is that some parts of 3D objects are missing due to 
the gap between SLMs. In this paper, we propose a time-division 
colourization technique for our previously proposed method of 
increasing the image size with no missing image segments by locating 
an optical system containing lens arrays and other components in 
front of multiple SLMs. Using the proposed optical system enables 
3D images to be colourized while enjoying the advantages that par- 
allax can be exhibited in both the horizontal and vertical directions, 
the balance between the image size and viewing-zone angle can be 
controlled, the device can be configured without regard to the ratio 
between the image size of each SLM and the gaps between SLMs, the 
image size can be increased by increasing the number of SLMs, and 
unnecessary light can be eliminated. 

We applied the proposed method to 9 SLMs to realize full parallax 
video holography at a frame rate of 20 fps with an image size of nine 
times that of one SLM and a viewing-zone angle of 5.6 degrees. 
Although the time-division colourization method had a frame rate 
of 20 fps because we used 60 -fps SLMs, we could also reach 60 fps in 
colour, for example, if 180-fps SLMs were used. 

When observing 3D image with focus on distant place from holo- 
gram plane in depth direction, boundaries of SLMs become diffuse 
and sparse, because these boundaries are exist on hologram plane. 
Actually, observed boundaries became sparser in Fig. 3 (d), focused 
on earth, than same one in Fig. 3 (b), focused on moon. 

There is difference of brightness depends on difference of contrast 
between each display area in Fig. 2 or 3. There is no difference of 
contrast between each SLMs, so it is thought to be dominant that 
difference of optical specification of each beam splitter, which 
composes an incident optical system. Each beam splitter provides 



different contrast, it becomes a bad effect for viewing 3D image. It is 
possible to estimate that we can balance among the difference of 
display brightness in each display area, if we make luminance com- 
pensation for hologram data, which is displayed on each SLM. 

Methods 

Figures 5 and 6 show configurations of used optical systems in this experiment. 

The proposed method makes multiple SLMs appear to be a single SLM with no 
gaps between individual SLMs, which are the hologram planes. Of course, it is difficult 
to physically eliminate the gaps completely. For an enlargement-type 2D display 
system in which 2D displays are tiled, since the viewer's eyes are focused on the 
displays, no matter how precisely the individual displays are aligned, the spatial 
frequency components of the interference caused by the boundaries of the displays 
are easily perceived by the viewer since they often contain low- frequency components 
relative to the video signals themselves. However, for a 3D video display based on a 
holographic method, the reproduced image can be displayed at a depth location 
separated from the hologram plane. In this case, the viewer's eyes focus on the 
reproduced 3D image rather than the hologram plane. If the interference caused by 
the boundaries of the SLMs can be suppressed enough so that the eyes do not focus on 
the hologram plane, it can be driven out to the Fourier transform region with respect 
to the viewer's sense of sight. In other words, since it is dispersed to the entire field of 
vision, it cannot actually be perceived. As a result, this method can be expected to 
actually suppress the interference caused by tiling the SLMs for a 2D display. 

Figure 5 shows the light source system that implements colourization based on 
time-division multiplexing also known as frame sequential method. This system 
consists of three lasers (red, green, and blue), wheel shutters, spatial filters, and a 
trichroic prism assembly. The shutters are rotating wheels driven by motors that are 
synchronized with the SLMs and controlled so that a beam of one of the three colours 
is selected to pass through the shutter every 1/60 second. After that, the laser beams 
pass through a spatial filter (convex lens and pinhole) so that the shape of the beams 
gradually becomes wider. Three beams are combined into one by a Phillips type 
trichroic prism assembly. Finally, the combined white beam is collimated by a large 
size convex lens to illuminate all of the SLMs. 

Figure 6 only shows an example of the horizontal direction for the case when the 
number of SLMs K x is 3. First, the previously described time-divided colour parallel 
light from a coherent light source as shown in Fig. 5 is incident perpendicularly on 



Parallel light 



> Convex lens 
Trichroic prism assembly 
y Pinhole aperture 
Focusing lens 




Laser Equipment — Y— 

Wheel shutter 

Motor 



Laser Equipment (i R ) 



Laser Equipment (A G ) 



Figure 5 | Configuration of colourized light source system. 
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Figure 6 | Configuration of optical system. 



SLM C° through beam splitter B°. Consequently, this is so-called in-line holography. 
The hologram data is displayed on C°. This data is synchronized with the wheel 
shutter and changed to accommodate the alternating wavelengths of the light sources 
(red, green, or blue) every 1/60 second. Note that the superscript at the top right of the 
letter represents the location within the array. Since the pixels are arranged in a grid 
pattern on a thin hologram, the light reflected on C° contains a primary beam, 
conjugate beam, carrier beam and high-order beams of each of them. Since the object 
beam is generated as a primary beam, the other beams are unnecessary light and must 
be eliminated. 

Next, the light is incident on lens Lq, spatial filter F° and lens Lj. Spatial filter F° is 
located on the common focal plane P : of lenses LjJ and Lj, and the focal length/ 0 of Lq 
and focal length^ of are related as follows: 



(4) 



depending on the image size S' of each SLM and the gap V between SLMs. Also, we 
set the distance a between C° and ll as follows: 



a = f f +/„. 
h 



(5) 



This enlarges the image due to C° by (S' + T')/S' and shifts it to Lj (P 2 ). 

Here, let us simply denote the group of C°, B°, Ljj, F° and Lj by a partial optical 
system A 0 . By locating partial optical systems A 0 and A 2 adjacent to A 1 , the images due 
to C°, C 1 , and C 2 will be tiled on plane P 2 without gaps. As a result, the missing image 
segments due to the gaps between the SLMs will disappear. 

The group of lenses Lq, Lj, and Lq forms the lens array L 0 in practice and the group 
of lenses L° v L}, and L^ forms the lens array L x . F° is a spatial filter for eliminating the 
conjugate beam, carrier beam, and high order beams, which are unnecessary light 32 . 
The group of spatial filters F°, F 1 , and F 2 forms the spatial filter array F in practice. 

Finally, the light is incident on lens L 2 and lens L 3 . By setting the focal length/ 2 of L 2 
and focal length f 3 of L 3 as follows: 



s 



S' + T 



(6) 



the image that had been enlarged at P 2 will be returned to its original size at plane P 3 . 

This optical system can exhibit parallax in both the horizontal and vertical direc- 
tions. It can also control the balance between the image size S x and viewing- zone angle 
6 X according to the values of/ 2 and/ 3 . By varying the value of/ 3 starting from/ 0 , you 
can configure the device with any ratio between the image size of each SLM and the 
gap between SLMs. The image size S x can be enlarged by increasing the number of 
SLMs K x and using a large-aperture lens for L 2 . Also, unnecessary light can be 
eliminated by spatial filter F°. Those are some advantages of this optical system. 

Proposed method has scalability to increase the number of SLMs for further large 
image size. There are two factors of size limitation. One is size of lenses, and the other 
one is vignetting of first order light by the outer edge of beam splitter, which was 
located at distant position from SLMs. A solution for latter problem is to reduce the 
number of steps of beam splitter to shorten light path in incident optical system. 
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